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^ (57) Abstract t The object of the present invention is to eliminate hitherto restrictive conditions related to photography like when 
g attempting to register pictures with infinite depth of field, more or less, or attempt full detail -reproduction of light and shade. This is, 

according to the present invention* accomplished by making several recordings with differing states of focus or exposure, the images 
O registered feeing similarly subdivided into smaller Image-parts and a resultant image Wng produced by selecting, those image-parts 

from each set, beiag most suitable. Optimal image definition of an image-part can for instance be assessed by contrast-measurement 
^ techniques or separate range-finding with a laser. An analogous optimal representation of image detail can also be achieved. 
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CAMERA THAT COMBINBS THE BEST FOCUSED PARTS FROM DIFFERENT EXPOSURES TO AN IMAGE 



5 

The present invention refers to a camera with an image registration device in its image plane,, 
preferably an electronic one like a CCD sensor. To be more specific, if a an electronic instrument with 
an objective lens, a pixel-oriented image detector with entrance Siirfac©> an electronic image memory 
for saving image information originating h the same detector and an automatio focusing device, 
10 according to a preferred mode of operation. The invention is furthermore referring to the 

corresponding methods, some of them applicable for (emulsion-) film cameras as well, however with 
a subsequent image process. 

Purpose of the Invention 

15 

The purpose of the present invention is to accomplish an instrument where hitherto restrictive 
conditions related to photography axe removed more or less, A major such limitation is the practical 
impassibility to produce photos of high image definition for all ranges. And strictly speaking, ifs 
equally difficult to attain short depths of field, suppressing image detail being outside this interval, 
20 such residual detail-blur manifesting another restriction. A third limitation of similar character, is 
associated with the frequently occurring situation of large intensity variations across a scene, usually 
in-between light and shade, making it impossible to register bright and dark areas h full detail. 

It may be recalled that depth of field for standard photographic lenses depends upon relative aperture. 
25 However a significant stop will also degrade die feasible image resolution, due to wave properties of 
light. Another consequence of stopping down an objective is loss of light which however may 
become less conclusive due to high sensitivity characteristics of modern electronic pixel-oriented 
image registration. 

30 To make pictures without being committed to these restrictions mentioned, has so far been reserved 
for the fine arts like sketching and painting more or less. This applies to depth of field, which is 
hardly regarded as a problem by an artist while scenes where well-lit and poorly illuminated or dark 
objects alternate, constitute another such example where some schools of the old classic arts excel 
It's a purpose of the present invention to gain a corresponding freedom for photographers. 

35 ' 
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Prior Art 

A method to combine diffenently-fbcused pictures in order to produce a compound image with 
improved image definition, is disclosed by US-patent 4,661,986: A video signal registration takes 
5 place for each state of focus for a three-dimensional scene and frequency-spectra related to these 
video signals support a comparison, row by row across the pictures, enabling selection of optimal 
pixels. Evan successively registered images with one and the same camera are described. A similar 
method and device is described in US-patent 6,201,899. 

Several camera designs with automatic focusing devices (Autofoous 1 ) and where the object Is to 

10 adjust for a best" image definition are known. One such design is measuring the time-lapse of an 
ultrasound pulse. Other systems work with contrast detection, where a focus-scan takes place and a 
setting for maximum contrast is assessed Tlie contrast is measured by two or several electronic 
sensors being mounted in a beamsplitter arrangement The adjustment is performed for a minor select 
part of the image, usually in the middle. Such a camera, denominated Konica C35AF, was introduced 

15 around 197? (c£ the periodical FOTO no, 1, 1978), Descriptions of similar systems may be found in 
American patents 4,078,171 and 4.078, 172. A4justment of focus, according to these cases, is optimal 
for one range only, while objects falling outside this plane (or curved surface, i.e. where sharp 
reproduction takes place, answering to the contrast measurement), become blurred more or less, 
depending upon die spread of object-distances. 

20 An entirely different objective lens design aiming at the movie industry and following US-patent 
4,741,605, makes it possible to focus upon two (or a few more) different distances at the same lime. 
One or several aperture lenses, are here inserted in order to cover part of the entrance aperture, Sharp 
image of a nearby object is created by placing this lens attachment in a favourable lateral position 
while the free aperture part is similarly depicting remoter objects, A drawback of this approach is that 

25 the ray path via the lens attachment is still creating a blurred* yet almost invisible, image of the 

remote object, while the other ray path through the unobstructed aperture creates a similarly blurred 
projection of nearby objects. These totally unfocused images axe therefore contributing with 
'background illumination', scattered over the image surface, thus as a consequence halving the image 
contrast more or less (i.e. for two images), 

30 As to the other problem with photos having pronounced lights and shades, there are few remedies 

known, except for using the most optimal positive/negative film material/sensor and to protect against 
stray light wherever feasible. Paper copies are problematic while slides do somewhat better. 

35 
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Summary of the Invention 

To sum up, this and other objects and advantages of an electrooptical instrument, designed in 
accordance with the present invention and described in the introductory passage or farther below, is 
5 attained by means of distinctive features mentioned in the descriptive parts of independent patent 
claims enclosed. Advantageous embodiments are further dealt with in the dependent claims. 
The invention may be summed up in brief albeit without restrictive purposes* as follows: 
Several exposures, equivalent to the same several pictures, are made with various eemero settings, 
Bach of these pictures are similarly subdivided into many sub-image segments or image parts. Each 
1 0 such portion of a scene is thus to be found in one set of image parts, albeit recorded with variable 
camera settings. One image part from each such group is furthermore selected and merged into a 
whole resultant image. The subdivision into image parts or sub- images may be performed in different 
ways and this is also true about the kind of camera settings which are to be adjusted in-between 
exposures. 

15 

Ifs obvious for a specialist scrutinizing this invention that a multitude of variations are possible. Like 
the subdivision into segments/sub-images, which can be firm or adjustable for numerous patters, And 
the several exposures, subject to subsequent assembly, may also be differently focused. 
Object distances can be measured directly with ultrasound or User-light for each image part 

20 Time-lapse or parallax measurements are practicable. Another approach is to measure the contrast 
within each sub-image, this being a preferred mode of operation. In principle no telemetry is here 
involved. ]£s nevertheless possible to estimate the range for each object-element projected, from the 
objective lens setting corresponding to maximal contrast, this constituting a special effect, enabling a 
sort of three-dimensional registration of the scene. The subdivision into image parts can also be 

25 effectuated as a function of the scene itself, this constituting another attractive set-up for the contrast 
measurement procedure: A kind of background or reference image (like the infinitely-focused 
exposure of a set) is here assigned, however parts with higher contrast are successively replacing the 
low-contrast areas as the process goes on. Less image memory is consequently required, this being to 
advantage. 

30 Various sophistication levels are conceivable when implementing the present invention. However 
technical solutions with minimal mechanical complications are sometimes at a premium, involving 
two or several fixed but differently-focuBed image sensory located behind some beamsplitter system. 
Other designs may include a mobile or abatable objective lens, viz, adjustable relative one image 
plane only. An undulating lens of that kind may prove less successful for cinematographic use 

35 however. A rotating disk with discrete steps of various thickness, may pdwide a better approach in 
that case* 
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No optimization of image definition nor focus takes place when photographing according to anotner 
modification of the present invention: Instead a camera records* preferably in fest sequence, several 
differently focused exposures. The subsequent image processing, including possible subdivision into 
sub-images, search for image parts with optimal resolution and final assembly of a resultant image, is 
5 now taking place in a separate unit, This modification is also applicable to other, non-digital sensor 
materials, like ordinary (emulsion-) film. 

Hie image contrast C in-between two adjacent picture areas of intensities A and B, is here defined as 
an expression like C = A-B/A+B where A is bigger than B, 

10 
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Brief description of Figures 

Pig 1 shows a digital camera with beamsplitter D and two differently focused image planes. The 
objective OB is projecting a scene F onto image planes Bl and B2 with associated image sensors 
5 CCD1 and CCD2. A processing unit P is receiving image information from the two sensors. It's 
dividing the images into small image parts or sub-images, selecting and forwarding those having 
superior image definition* to memory M. 

Fig 2 displays a surveillance camera application 

10 

Fig 3a is displaying an arrangement -where focusing is effected by means of moving an 
objective lens. 

Pig 3b exhibits a similar design, except that range-finding applies to each little image part, this being 
1 5 decisive for a choice of optimally focused segments) from each set, thus no contrast measurements 
taking place here. 

Fig 4 shows another digital camera with objective lens OB, a variable illumination-limiting device 
(like an aperture stop) VS and an electronic sensor CCD registering images. An exposure meter E is 
20 furthermore dividing the image into parts i, i+l, i+2.. .which are individually and differently exposed. 
There is finally an electronic image processing unit P which is, among other things, restoring and 
adjusting for final light intensities, as visible at the presentation (or memory) unit M. 

25 
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Examples of Embodiment & Application* in detail 

The present invention applies to an electrooptical instrument with capacity to eliminate some of the 
more fundamental restrictions which have always prevailed within photography. Most examples 
5 exemplifying the invention here aim at depth of field-improvements. However, other fundamental 
limitations may be remedied m an essentially similar or equivalent way, 
If* an instrument with capacity to measure image definition, optically projecting a scene upon an 
electronic detector, registering images/frames in different states of focus, and all subdivided Into 
smaller parts. That part-picture having beat image definition is chosen and merged into a resultant 
1 0 final imag$> being saved in an electronic memory and/or being presented as a picture on paper or an 
image screen. 

The invention, according to various configurations in the characteristic parts of relevant olaims 
presented here, is thereby solving the problem to obtain high image definition for various focal 
distances, on one and the same image, 

15 

The most important design parameter for an image-registering instrument, may well be highest 
possible image resolution, Le. to produce a sharp image over largest possible portions of the total field 
of view. However, strictly speaking, sharp images are only created for objects staying in an optical 
focus, while objects out of focus* within the field of view, remain more or less blurred, which may 

20 often be to disadvantage. For example a soldier using night vision goggles (Cf. Patent SE45067I) 
may stumble against nearby branches, barb wire etc because ho can only focus on one distance at a 
time and small nearby objects cure thug wiped out, because the device is mostly focused further away 
due to other practical reasons. Another example may involve an automatic surveillance camera, where 
the task is to identify persons and objects at various distances at the same time but where only one 

25 focal distance at a time, is feasible. Even a still camera photographer may experience problems 

associated with various object distance^ far example when attempting to take m indoor photo full of 
details, showing the distant wall as well as nearby objects, in high resolution. And the need to focus 
on two actors at the same time, happening to be at differing distances from the camera, is a problem 
from the film industry. A remedy for this last problem has been suggested (C£ US Patent 4,741,605) 

30 as follows: A movie film camera lens aperture is divided into parts in such a way that two differently- 
focused but superposed Images are created. However, consequently the two images in fbcus are also 
merged with two other images out of fbcus, thus halving the image contrast more or less. This method 
does furthermore only provide two focal distances while a normal field of view may be built up of 
objects with several more states of focus, some of them even rapidly changing/moving. The effective 

35 F-number of the instrument is also influenced by this technique. 
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The present invention does improve this situation inasmuch that many more focal distances can be 
used and unfocused/blurred image ^formation is furthermore rejected so that the final image is 
mostly containing high definition and high contrast contributions. 

Thus, an instrument according to the present invention, with capability to refocus continuously for all 
S distances iu-betweau infinity and closest range more or less, then register and sort the image 

information as described, should be able to produce high image definition all over the fh)al image. Or 
in other words, an instrument according to die present invention is de facto producing images with 
infinite depth of field 'Depth of field 1 is a commonly recognized measure for the distance interval, 
centred around an associated focal distance, within which a photo remains sharp. A short such 

10 'in depth 1 distance is equivalent to poor depth of field, being degraded by working with high speed 
lenses (Low F-number) and large optical entrance apertures or long focal lengths in general* like 
telephoto lenses. The main object of the present invention, i,e. to improve the depth of field, is by no 
means a new thought: The traditional mode of improving optical instruments in this respect has been 
by decreasing objective lens diameters, like stopping down a camera lens, cf. above. However as a 

1 5 result, the lenses gather less light, implying other drawback* like longer exposure times, giving 

associated motion blur and grainier film, and these effects degrade the definition of a final image. The 
objective lens diameter may even be reduced to the sfoe of a needle point aperture of a so-called 
Camera Obscura* with the capacity to project images with almost infinite depth of field, however 
unfortunately increasing the photographic exposure times to hours or days at the same time, making 

20 this method practically useless for most applications. 

Another well-known remedy for depth of field problems, is to miniaturize the instrument, i.e, design 
for a shorter aystem-fbcal length, like when Leica introduced the new and still prevailing 35 mm 
miitic&inera& Another similar development has taken place more recently for those small digital 
cameras having only about 6x8 mm sensor-size and a normal focal length around 12 mm, thus just 

25 one quarter of the equivalent minicamera, where the frame size is 24x3 6mm and focal length is 
around SO mm. Even the image intensifier technique undergoes a similar development, with linage/ 
photocathode reductions from 25 mm or bigger m the 1960-70's to 18mm during the 1930-90's and a 
farther reduction to 12 or 16 millimetres today in USA and Europe (Cf. US Patent # 6,025,957). 
This miniaturizing gives an improved depth of field in general, however at a cost of reducing fhe 

30 number of image points/pixels being resolved across an image area. This development was 

nevertheless made possible by an equivalent improvement of image resolution and light sensitivity of 
the registering components involved (like CCD sensors, linage Intensifier Tubes and Photographic 
film). These above-mentioned methods offer same relief but the depth of field problem is still there. 
One professional category only, namely the landscape-painters, have from time immemorial been 

35 able to master these problems, by using the oldest known optical device (the eye): Sometimes 

painting the nearby foreground, sometimes the background but reproducing each little object by itself, 
thus assembling a whole painting from a great many differentfy-fooused image parts. The later 
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observer may therefore (apart from some more artistic qualities) appreciate the 'infinite depth of field' 
of the old paintings: A painting is watched from a certain distance, even though teal objects from 
widely vaiyiog ranges are reproduced. This artistic way of metamorphosing a scene, with entirely 
different (optimal) states of focus, into a flat and everywhere sharp reproduction and with one state of 
focus only, has certain features in common with the present invention. 

The above-mentioned depth of field-problems can be eliminated or at least reduced by utilizing some 
characteristic features related to the present invention, following the introductory passage and 
exemplified below by an electrooptioal instrument like a video camera, a digital still camera, en 
image intensifier instrument or a surveillance camera (i.e. Instrument* for short); 

L The instrument is provided with an automatic focusing device (1) so that the 

objective lens (2) may be focused on more than one object distance. 

2. The initial image Bl is focused on a suitable distance, like 'infinity. 

3. The same image is registered by the detector (3), thereafter transferred to 
an image memory (4). 

4. There is an image-part-function (5) associated with the image memory which 
subdivides the same whole image into smaller sub-image segments Bli, Bl(i+1), • 
Bl(i+2)...thus making it possible to address these image parts, making them 
individually accessible from the detector and/or image memory. 

5. The instrument is next focused for another and usually pre-determined object 
distance and a more recent image frame B2 1b registered by the detector. 

6. The Instrument is also incorporating an image-definition meter (6) with capacity to 
assess the image resolution of each little sub-image individually. 

7. This image definition-detector is associated with a comparison-function (7), 
enabling comparison of image resolution for each sub-image couple, i.e- Bli with B2L 

JL Initial image-part information Bli is replaced by corresponding subsequent 

sub-image information from B2i, in case image definition of B2i is estimated to be 
superior to Bli; 

Or alternatively, later image information of B2i is discarded while previous 

information from BU is retained without alteration of memory, if the opposite situation 

occurs, i.e. the sub-image B2i appears to be less in focus than Bli. This selection 

procedure is repeated for all image parts 1^3,. ,L 
9. The Instrument is thereafter resfocused again, more pictures B3, B4-. JBn being 

registered in the process and the same procedure (#5-9) is repeated. 
10* The resultant image (i.e. at least as far as depth of field-enhancement procedures 

of this invention goes) is finally in memory, when the last focusing step has been 

finished 
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The simplest design of this 'instrument 1 involves an objective lens (2), however other optical system- 
components like teleconverters, eyepieces, scanning- or (image) relay-systems may be included for 
certain applications, making the total system more complex. Automatic focusing devices (1), (not to 
be confused with the well-known 'Autofocus^ see below), can be set up in many different ways: Like 
5 in time-sequence, so that different focuses appear in succession. However a static mode, for example 
by means of beamsplitters, is also feasible, involving several active and diflBerently-fbcused image 
planes at one and the same time. There are detectors (3) of various kinds, but the so called CCD chip, 
made up of a two-dimensional matrix of pixel-sensors, is most frequently occurring in video-' and 
digital still cameras. There are also infrared (IR; like pyruelectric) sensors, vidicon- and image- 

1 0 hrtemsifier tubes. The detectors may also be singular or linear detector air ays* Image memory (4) is 
here a wide concept covering electronic computer memories associated with the instrument, magnetic 
tapes, RAMs, hard- or floppy disks plus CD or DVD disks and 'memory cards', commonly delivered 
these days with digital cameras: This latter kind is constituting a final memory for an Instrument, like 
would also (sort of) be the case for an image printing process, where digital information may cease, 

1 5 image information nevertheless surviving on the photographic paper. And associated with this are 
presentations on image screens for TVs and computers and other image viewers or screens which 
only retain an image as long as the presentation lasts. It may prove advantageous for some 
applications to use several memories, like for the image process inside an instrument plus a final 
memory where only processed images are stored. The pictures Bn are subdivided (5) into image 

20 segments or sub-images Bni 5 eaoh of them (if applicable* see below) big enough for some contrast 
measurement, however still small enough for ensuring continuity and -uniform image definition across 
the final picture* The instrument must therefore incorporate an image definition-meter/analyser (6) to 
bring this about, like a passive contrast measurement device of the kind prevailing in video- and still 
cameras since long ago. The first introduction of such a camera on the market was possibly by the 

25 manufacturer Konica with its 'Konica C35AF camera (C£ an article in the periodical FOTO 1/78% 
incorporating an electronic range-finder, founded upon the principle that maximum imagp contrast 
and resolution occur simultaneously more or less. The focal distance for a small picture area in the 
central field of view was thus measured with this camera through a separate viewfinder, identifying a 
state of focus with optimal image contrast, thus approximately answering to the best resolution, 

30 whereupon the lens of the Konica camera was automatically refocused accordingly. This is the 
common method even today more or less, cf, for example the Olympus digital still camera 
C-300ZOOM, having a somewhat similar autofocus device according to its manual. 
There is an important distinction though, as regards how this contrast measurement technique is 
utilized, according to the present invention; While the abov^mentioned commercially available 

35 consumer cameras are adjusting for a best focus using this technique, the veiy opposite takes place for 
instruments, according to the present invention: Instead a sequence of images with pre-defined states 
of focus are exposed and the contrast measurement technique is applied afterwards in order to select 
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(7) the sharpest sub-images And there is another fundamental difference: Contrast measurements, 
according to the present invention, take place all over the image/field of view, while Auf ofocus 
cameras according to present day technique, are mostly measuring inside a small image-segment 
only. It may thus be asserted that an instrument incorporating the present invention, may well use 
elements of prior art, but in an entirely new context. Explicit range measurements are not necessitated 
by this technique, however if s feasible to assess average distances for each image segment because 
optimal states of focus and thus (in principle) the appropriate focal distances are known by means of 
this contrast measurement approach. The introduction of a distance measurement Amotion of this sort 
provides the basis for continuous mapping of projected scenes in three dimensions, because the 
information of each sub-image segment (Co-ordinates X and Y) is now associated with a distance Z 
(Co-ordinate in depth). It would therefore be possible to transfer this image and distance information 
to a computer, the object for example being to produce three-dimensional design documentation for 
the scene depicted, thus the basis for applications like 3D presentation, animation etc, A small video 
camera can be moved inside reduced-scale models of estate areas, within human vascular systems, or 
inside the cavities of machinery, sewage systems or scenes which are to be animated for film or 
computer game purposes, not to mention industrial robots requiring information about all three 
dimensions when maneuvering its arms: All these applications mentioned may, as a consequence of 
the present invention* benefit from the continued supply of three-dimensional information, related to 
each image part of the scene. 

As a matter of principle, ifs possible to cajty out the distance- or image definition measurements on 
one occasion only, for a stationary instrument, i.e. same aiming all the time, involving a static scenery 
more or less. This application is explicitly exemplified by installation of a stationary surveillance 
camera as follows: The camera undergoes the above-mentioned process #1-10 during installation and 
this is possibly repeated after each fresh start up. It r s preferably done very precisely, by means of 
many nearby states of focus, the whole procedure being repeated a number of times so that an 
average of many cycles #1-10 may be estimated, the optimal focus of each image part thus being 
assessed more precisely than would otherwise be the case, 

The camera may henceforth be operated without necessarily using image deflnition-ineasurements, 
because the fixed and essentially stationary scene ranges are already known* the most optimal states 
of focus for each image part thus remaining the same more or less, being saved in a memory. 
Temporary and stochastic disturbances, like waves on a sea or swaying trees at the homon, may 
furthermore influence wide areas of a fixed scene during stormy days, thus affecting the image 
definition meter. A better solution would be to save this above-mentioned range-finding procedure for 
some calm and clear day without that multitude of fast flutter. 

A Sequent and major task for surveillance cameras is to detect and observe new objects, figures etc 
emerging on an otherwise static scene. Such objects may or may not emerge at the same, static/initial 
object distance from the camera, thus appearing more or less blurred, depending upon current 
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depth of field and other parameters, in case the image definition-detector was switched off. However 
turning this meter on again t it would be possible to detect new objects within the field of view by 
comparing the initially assessed states of focus for each sub-image, with any more recent such 
measurement, thus enabling detection of changes within the field of view, Le* for each specific 
5 sub-image segment, causing the alarm to go (blinking screens* alarm bells etc). 

The function of an image definition-meter may involve some algorithm for the assessment of image 
contrast (Cf. US Patents 4,078,171 and 4,078,172 assigned by Honneywell) within a small sub-image. 
Let's suppose this is done with n detector elements, uniformly distributed over the sub-image. At least 
two such detector elements are necessary for the contrast measurement: Suppose an (image) focus- 
1 0 edge is crossing this segment: A bright sunlit bouse wall (with intensity Lmax) being (for example) 
registered by detector D 1 on one side and a uniform but dark background (intensity lmin) like 
thunderclouds at the horizon, being registered by detector D2 on the other side. The contrast may 
then be written as 

15 Cmax = (Lmax-Lmin)/(Lmax + Lmin) 

according to elementary theory. There would ideally be these two light levels only, as long as the 
house wall is in focus, Le. the edge is sharp. However the edge becomes increasingly blurred if 
defocusing fhe instrument, i.e. light intensity will then gradually (depending upon the distance 

20 in-between the two detectors) change from Lmax to Lmin when passing a transition zone in-between 
house and background. The intensity measured by detector D L will thus decrease from Lmax to LI 
while detector D2 is registering a light intensity increase from Lmin to L2. The difference L1-L2 and 
as a consequence the contrast C - (L1-L2)/(L1+L2) are thus diminishing. 
And it T s vice verm possible to identify and correct for such image intensity gradients of a picture by 

25 means of electronic image-processing programs, compressing the transition from a bright to a dark 
zone, so that fhe gradient is increasing, the border in-between darkness and light becoming narrower 
and the image consequently getting sharper. 

An image definition and analysis function associated with the present invention, should ideally 
choose that state of focus corresponding to the close house wall of the above-mentioned and much 

30 simplified case, thus giving a sharpest possible edge against the dark background. However a 

significant further contrast structure of the background would complicate matters, creating another 
optimal focus within the sub-image segment A generalized contrast algorithm involving more than 
two detector elements would then be required. A further development of this method is to replace 
above-mentioned step #8 with an alternative and expanded procedure, where image definition and 

35 information, registered and measured for each image part and for each state of focus during a 
focusing-cycle are saved, and this would make it feasible to choose and perform some kind of 
weighted fusion of image information, related to several optimal states of image resolution. The 
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statistical weight of a corresponding major maximum migfrt even be chosen as zero, nice tor tne 
feasible case of a surveillance camera being directed through a nearby obscuring fence. A new 
dstance-discrtminatory function would be appropriate for such oases, i.e. a device blocking out image 
parts with optimal focus closer than a certain proximity distance* like the above-mentioned fence. 
5 Another example: The Instrument may be focused for two optimal states (other focusing distances 
being blocked out) for every second final image respectively, beir\g produced. 
A typical case would be a nearby thin and partly transparent hedge, through which a remote 
background is visible. 

If s obvious that a comprehensive catalogue of all applications implicated by the present invention, 
10 can't be presented here. It's emphasized though that characteristic features of the invention, according 
to the claims enclosed, are of general validity and the exemplary embodiments shall not limit this 
scope. 

Another and essentially different image definition measurement method is involving actual distance 
measurements with for example a laser range-finder: This is an active method, similar to radar* 

1 5 involving a laser pulse transmitted, then reflected against a target, finally returning to the detector of 
the laser range-finder receiver. The distance is calculated from the time measured for the pulse to 
travel forth and back. This procedure must, according to the present invention, be repeated for the 
individual sub-images and one way to bring this about is to let the transmitting lobe of the laser 
range-finder scan the image vertically and horizontally, somewhat similar methods for instance being 

20 employed already in military guidance systems. The laser range-finder transmitter and receiver can be 
separate units or integrated with the rest of the optics and structure of the instrument. For example, it 
would be possible to design an Image detector (3X where each little segment (besides registering the 
image), is incorporating a laser detection-function, thus integrating the range-finder receiver with the 
image recording parte of the optics, reJated to the present invention. The distance to, and as a result, 

25 optimal state of focus for each image part may thus be assessed because focal distances related to 
pre-detertoined states of fbcuB are known, in principle. No explicit measurement of image definition 
is thus required here (c£ F*g 3b). The distance information does nevertheless point out those 
differently-focused image*? arts, which are offering optimal image definition. 
The object of bringing forward these techniques, ie. how to assess optimal states offceus for 

30 sub-images wfthin a scens, is to demonstrate the wide-ranging feasibility of the present invention, 
rather than restricting its scope. 

Only part of the image processing required has so far been covered by tills text Further steps may 
prove essential for a more satisfactory outcome of the operation: If s an Important design target that 
the instrument is capable of producing uniform pictures without visible discontinuities. Choice of 
35 sub-image size may turn out crucial, it's for example important to make them small enough in order to 
secure optimal continuity in-between adjacent sub-images as regards state of focus* On the other 
hand, sub-images must be made big enough in order to resolve image structures, like contrast 
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variations, with enough precision with reference to the above-mentioned contrast measurements. 
Spatial smoothing of adjacent image segments, is another well-known technique, c£ for example 
existing Adobe Photoshop commercial (PC) programs. This procedure may improve image- 
uniformity, however a* the same time tending to degrade the contrast of minor image details. Image 
5 compression techniques like * jpg may also be incorporated but this is not the proper forum for 
repeating facts about well-known techniques. 

A novelty though, related to the present invention, is that averaging of image information may be 
expanded from the 'normal 1 X/Y image plane to the third in depth dimension Z, involving a4jacent 
state of focus for one and the same image segment this however requiring adeepjate storage memory 

10 for such running image information. 

Another pbenomento be considered, is variation of picture size in the process of refocusing, like 
distortion (Le. magnification variations). However an electronic remedy for this is possible, like 
keeping sub-image sizes unaltered, irrespectively of actual state of focus. 
An essential aspect of the invention is thus that the instrument can he appropriately refboused, a 

15 subsequent choice in-between different states of focus thereafter taking place. 

The modus operandum may be static by means of partition into several image planes, but more 
generally dynamic by following an automatic pre-defined time-sequence schedule, and there is a 
multitude of different ways to bring this about 

One common method to focus a camera is for instance to move one or several objective lens- 
20 components, usually at the front, along the optical axis. A single continuous refocus-movement from 
infinity to - say - the proximity distance of a meter, can he executed in this way. This refocu&ing* 
process may thus take place continuously rather than in discrete steps which may prove advantageous 
at times. However, these mobile lenses must stop at the ends, the motion thereafter becoming 
reversed, which may prove impractical at high speeds, and where many focusing-cycles per second is 
25 an object. The method will nevertheless suffice where refoeus-frequency is low, like for certain 
digital still photo cameras. 

Another method would be to introduce one or several glass-plates of different thickness, usually 
in-betweeu exit lens and image plane. Such glass plates are extending the optical pathway* moving 
the image plane further away from the lens. Several such plates of various thickness, placed on a 

30 revolving wheel with its rotation axis differing, albeit in parallel with the optical axis, may be 

arranged so that each of the plates is> one by one and in fast succession, transmitting the rays within 
the optical path, as the wheel rotates: This is a very fast, precise and periodic refbeus-procedure 
and it would be possible to rotate a small lightweight low friction-wheel with a uniform, yet high 
speed of at least - say - 1000 turns per minute. This mode would therefore approach TV application 

35 speeds more or less, with 25 pictures pet second (PAL). Each picture should be registered and 
processed for the different states of focus, Le. ideally 5x25=125 pictures per second here, with 5 
different states of focus (like infinity 10, 5, 3 and 2 meter focal range). However, a trade off 



i 
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involving rotation speed reduction and several TV frames, associated with one fbcusing-cycle only, 
seems feasibly though possibly causing side-effects as the differences in-between consecutive TV 
frames mount up, and for fast moving objects. 

The fastest method however, should be to equip the instrument with several difteremtly-fooused 
5 image-detectors. It's for example an established digital video camera technique to use 3 CCD-sensots 
(like the Sony DCR-TRV900B PAL camera; The purpose in this case being different however, 
namely to register the three main colours (RGB) with separate sensors), There are several ways to 
implement this in practice like inserting suitable beamsplitters, usually close to the image plane 
behind the objective lens. A split into two or several spatially separated images can be arranged by 

10 such means and each of these pictures can be registered with for example CCD-sensors. Beamsplitters 
are in common use and may be made of dichroic or metal-coated mirrors or prisms in various 
configurations and with differing spectral and Intensity characteristics depending upon requirements 
for specific applications. The advantage of this procedure with reference to the present invention, is 
that it gives simultaneous access to a sequence of pictures* only differing about state of focus. The 

1 5 comparison procedure may thus be undertaken with pictures having been registered at the same time 
and all time-lag influence caused by successive refbeusing h avoided. The method is apparently only 
feasible for a few, like three, detectors, i.e. states of focus, which may hamper certain applications. 
Refbcusing-cycles with many steps of focus may reduce the effective speed of an instrument, or 
prolong the exposure procedure; Assuming the total time for such a cycle to be = t and the number of 

20 steps/frames to be = n, then the effective exposure time* i.e. time available for registering each state 
of focus, becomes *■ tfn> A total exposure time, associated with the final image, must therefore 
(exposure conditions being the same) become n times longer than for the single exposure of a 
standard still camera of today (i.e. this invention not being applied). The consequences may well be 
negligible though, or comparable to those time-lags introduced already by autofbeus and flashlight 

25 functions* Fast objects may move a little in-between successive exposures however, motion blur is 
essentially originating from single frames, because only one state of focus is relevant for focusing 
upon the majority of objects. 

Three modes of focusing have been described so fer They are representative but there are many other 
ways to bring it about, exemplified in brief as follows: 

30 The detector may be focused by axial translations, being small most of the lime like tenths of a 
millimetre, but still an oscillation forth and back which may be impractical for feat sequences, at 
times. A most interesting concept would be a three-dimensional detector with capacity to detect 
several dlffbrently-focused 'in depth 1 surfaces at the same time. Thus no mechanical movements nor 
beamsplitters necessary here whatsoever, though the costs may be of some consequence. 

3 5 The above-mentioned wheel can be replaced by some rotating optical wedge giving continuous 
refocusing but introducing optical aberrations at the same time: It may be acceptable though, or at 
least possible to correct, 
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A multiplicity of technical solutions aiming at lefocusing-prooedures necessitated by the invention 
are thus available. The principles of the invention are of course not restricted by these particular 
examples. 

A particularly simple application example (Fig 1) of the present invention, shall now be described, 
5 where memory capacity requirements and mechanical movements are minimal. The objective lens is 
projecting an image of the field of view F on two image planes B 1 and B2. This split is done by a 
beamsplitter D, dividing the wave-front into two different parts with equal intensity. The image plane 
Bl is here stationary and the image is detected by the CCD- sensor CCD1 while the mobile image- 
plane B2, corresponding to various states of focus, can be detected with another sensor CCD2, which 
10 is subject to axial movements. This adjustment - dR may be effected by turning a knob on the outside 
of the camera, corresponding to refocusingthe instrument from infinity to - say - the proximity 
distance of a meter. The two detectors are connected to an electronic processing unttP, with the 
following functions: 

IS 1. Images Bl andB2 are subdivided into sro&U image parte BH and B2i 

by electronic means, 

2. Image contrast (sharpness) for each image couple Bl i respectively B2i 
b calculated 

3 . These contrast values are compared, i.e. for each couple. 

20 4 . Sub-image information associated with that image part (from a couple) 

having superior image definition, is forwarded to image memoty M 
(Information from the other image part being rejected) 

This procedure is repeated over and over again, for all sub-image couples Bli/B2i, Bli+LB2i+l„, 
25 until a resultant final image has been saved in image memory M, which could be a so called memoiy- 
card, detachable from the camera after finishing photography: Such cards are nowadays widespread 
for digital still camera use. Some further image processing like * jpg compression, distortion- 
correction and edge smoothing in-between a^acent image segments or sub-images (of. above) may 
also take place in a more realistic scenario, where additional memory capacity may prove 
30 advantageous or even neceseaiy (c£ Fig 1 ) for intermediate storage of image information while the 
process is going on in the processing unit P, c£ Fig 3 . Image elements from two different states of 
focus only, are thus contributing to this particular final image, however the associated depth of field- 
improvement is still significant: Suppose the focal length of an objective camera lens OB is around 12 
millimetres, other parameters like F-number and ambient light condition being reasonably set The 
35 depth of field could then well be from infinity down to something like 5 meters for sensor CCD1 

where the focal distance is - say ■ 10 meters, Let's furthermore suppose that the second CCD2 sensor- 
focus is set at 3 meters, creating a depth of field from - say - 5 meters down to 2 meters. The total, 
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using both detectors* would then be to create an accumulated depth of field in-between infinity and 2 
meters* as manifested on merged and final images, viz. after having applied the methods of the present 
invention, This is of course much closer than the five meters, however ifs only one of numerous 
hypothetio examples. 

A stunningly fast development of digital still camera performance is presently taking place, which 
will thenceforth accentuate focusing and depth of field-issues. The Olympus-camera CAMEDIA E-l 0 
with 4 million image-pixels and flexible image processing may represent a 'State of the Ait 
(i.e. in AD, 2000). 

It's also possible to move the image process, associated with the present invention, to a separate 
computer (PC) station. The processing of differently-focused image sequences from one and the same 
scene, moreover registered with some standard camera without processing-refinements prompted by 
this invention, may take place there. No electooptical instrument, like a digital camera, is necessarily 
required for this mode of operation, A traditional emulsion-film camera will do, imago digitizing 
possibly talcing place in a scanner, after the film has been developed, subsequently forwarding the 
frames to the computer. This may for instance be done in the processing laboratory engaged, ensuring 
that depth of field-enhanced photos are delivered to customers, who don't even have to think about it! 
Basic principles of the invention remain the same nevertheless, i.e. frames must stilt be converted for 
a digital/electronic medium in order to have them processed, However two main ingredients of the 
invention, i,e. a focus-scan followed by some digital processing, aze here separated, the latter taking 
place independently and somewhere outside the camera proper* Instruments associated with the 
present invention, are thereby physically generalized, now involving more than one locality like 
(in this specific case) a still camera and a PC computer with requisite program software. 
As a contrast, the already described stationary video surveillance camera provides a more 
complex system and what is more, may incorporate image intensifiers (i.e. nightvision capacity) and 
telephoto lenses. Crucial lack of light and poor depth of field in association with large apertures 
(small F-number) and long focal lengths, may here arise. Ifs possible to increase the memory capacity 
of the system, enabling storage of image information and focusing data from frames belonging to 
several focusing cycles. Processing and selection of image information may then be more 
independent of focusing cycles, allowing introduction of delay and a certain time-lag in the system 
before the processed images are presented on an image screen or are saved an magnetic tape or DVD 
disk Image processing may even take place much later in another context and somewhere else, 
using for instance magnetic tapes with primary information available. This is similar to the still 
camera example (above) where the basic function was split into two or several spatially separated 
embodiments: The Image registration involving lefocusiog-cycles is thus accomplished by a 
surveillance camera in situ while the image process may take place miles away in a computer. This 
procedure allows for the use of more powerful computers, a possible advantage where huge volumes 
of information are to be handled. 
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This application shall now be described in more detail (Fig 2); The surveillance camera is installed at 
locality A > where the scene F is projected by objective lens OB onto an image plane where a CCD- 
sensor belonging to a video camera is detecting the image. A focal fluctuation in-between four 
different states, is executed with the focus-wheel FH incorporating four flat glass-plates of different 
5 thickness: If s a fast revolving wheel giving four different focuses pea: torn. Video frames are 
registered on the magnetic tape/video cassette T at recording-station R. This video tape T is then 
transported to another locality B somewhere else, where the tape T is again played on another video 
machine VP forwarding image information to a processing unit P, which is selecting that better- 
defined image information in focus* already described (above). The processor P is therefore, in this 

10 specific case, selecting information in focus from image groups of four, The processed video film is 
finally stored in memory M or presented on image screen S. A more qualified use* under poor light 
conditions in particular, may involve the record and presentation of raw unprocessed images as well 
as depth of field-enhanced images, following the principles of the present invention. Optimal 
focus ing-data may moreover be stored for respective image-parts, thus avoiding to make contrast- 

1 S measurements all the time, this being particularly expedient when such measurements tend to be 
ineffective or erven impracticable to undertake, like whenever light conditions are poor. Other 
functions belonging to this kind of somewhat sophisticated systems* may include an option to vary 
the number of sub-images employed or the number of differently focused frames during a cycle, the 
object being to reach optimality for various ambient conditions* 

20 Certain aspects of the present invention are further illuminated and exemplified in Fig 3 a as follows: 
A view F is projected by objective lens OB onto a CCD-sensor, This lens OB has a mobile lens 
component RL, adjustable (dR) along the optical axis, equivalent to refocusing from infinity down to 
close range. The lens RL is moving forth and back in-between these end stops, passing certain 
focusing positions where exposure of pictures take place in the process. Image information from such 

25 an exposure is registered by the sensor, then forwarded to a temporary image memcry TM1 . The 
processing unit Pc is capable of addressing different sub-images and to receive selective sub-image 
information from TM1 and similarly from the other temporary memory TM2, the latter containing 
optimal image information, previously selected (hiring the foeusing-oyole going on, Image contrasts 
are calculated and then compared for the two states and that alternative giving highest contrast is kept 

30 in memory TM2. Even more information may be saved in memories like TM3 (not shown), speeding 
up the procedure further whenever, as a consequence, certain calculations (of contrast for example), 
do not have to be repeated over and over again. Further image processing, where the object is to 
improve upon image-quality and possibly oompress the image, will then take place in unit BBH and 
the resultant image is ending up in final memory M. 

35 The situation in Fig 3b is similar except for one important thing: The processing unit Pe is no longer 
calculating image resolution nor contrast Instead the processor gets its relevant information about 
optimal states of focus for different sub-images from other sources, Le. memory unit FL This 
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information may originate from a laser range-finder or be range information earlier assessed ttom a 
stationary installation (of* above). Such information suffice for the processing unit Fe when selecting 
image information for each state of focus, giving sharpest possible image. This select information is 
finally transferred to the temporary memory TM2> the rest of Hie procedure following Fig 3a (above). 
Various possible set-ups and applications related to the present invention and its depth of field- 
improvement functions have now been proposed. It would be possible to describe numerous other 
such explicit versions having the following features h common: 

1 > It's an eleotrooptical instrument; in the sense that original piotures are 

projected by an optical device while electronic digitizing and image 
processing associated, is taking take place in the same physical embodiment 
or somewhere else. 

2. This same electrooptical instrument; i.e. described by claims and text 

associated with the present invention, can distinguish and detect individual 

sub-images or segments of a whole image. 
3 k The image may be refbcused in various suitable time-sequences. Or as an 

alternative, the instrument is capable to register more than one differently* 

focused image simultaneously. 

4. The instrument may select that state of focus corresponding to a certain 
set of (similar) sub-images, giving optimal image definition. 

5. Image information from the most optimally focused frames, b elonging to 
each individual sub-image set, is added to a final compound image being 
effectively assembled from differently-focused image partB more or less. 

6. The resultant image is saved in an appropriate final memory and/or is 
presented on an image screen or similar. 

Image processing 

The image information required is, according to the present invention, extracted and assembled from 
original exposures, depicting the same scene, but with different settings. The object is to produce an 
improved final image of select image information and this can be achieved in several different ways, 
described as follows and commencing with methods related to improvements of depth of field. 
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I. Average-methods 

The mode of superposing differently-focused images is conceptually simple, It may be 
accomplished as follows: 

5 

1. Optically in an objective lens system or a binocular, fay means of dividing the aperture 
in portions of different refractive power (of. American Patent 4,741,605 byAlfredson etal) 

2. By means of a wave-front division in part reflective mirrors, thus generating at least 
two ray paths, which are spatially separated and differently-focused^ however finally 

1 0 reunited into a composite image, made up from differently-focused contributions. 

3. By refocusing an objective lens, belonging to an instrument for image registration, 
this being so quickly executed, that several states of focus occur during the exposure time. 

4. Periodic refocusing, faster than the physiological reaction time for the eye (around 1/10 
of a second) in visual instruments like optical viewfinders or telescopes, so that the observer 

15 is unable to perceive individual images, this being rafter much like watching a movie. 

5. By double exposure of a Classic' emulsion-film camera and 

6. Electronically by means of some superposition or pixel by pixel summation of differently- 
focused electronic images. 

20 The feature in common for these average-methods is some summation of all available image 

information, thus including the out of focus contributions as well, however thereby degrading image 
contrast and quality in the process. Such images are here denominated 'average images' (M), with the 
corresponding 'average-method' for short (Reference: Program software for PC computers for the 
pvKpose of superposing electronic images exist, c£ 'Image Fusion Toolbox' for taatlab 5.x.', now 

25 freely downloaded from Internet address www.rockin@er.piirespace*deftooibox_r.btm; Cf. also 

reference to a linear superposition 1 average-method correspond log to #6 above, on the web address of 
same Oliver Rockinger, and his thesis 'Multiieeolution^Verfahren zur Fusion dynarnischer 
Bildfolgea', Tecfamsche University Berlin 1999, for a mare general account of image fusion methods. 
The digital camera 'Finepix S IPro 1 , produced by Fuji Photo Film Co Ltd in Tokyo Japan, allows 

30 superposition of sequentially exposed images). 

2. Contrastenhanced average methods 

A further developed and improved method, related to electronically registered images, is involving 
35 an additional procedure of subtracting or removing the above-mentioned out of focus insage- 
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information. The result may generally be described as a concentration of 'focused image information' 
in the final picture or in other words, out of focus -image information is discarded, This process may 
be more or less efficient, depending upon model approximations. A version denominated 
'contrast-enhanced average method' will be exemplified as follows: 
S The above-mentioned average image (M) is defbcused, its intensity thereafter being reduced by a 
suitable footer and this picture finally being subtracted from the compound average image (M). This 
last procedure implies a de facto reduction of noise from the average image (M), this being the 
purpose. The above-mentioned defocuaing may be performed electronically, such 'blur-functions' 
generally exist in commercially available image processing programs (like the Thotoshop' PC 
10 programs from Adobe Systems Inc, USA). A 2-image process may thus symbolically, and in a very 
simplified way, be mitten as fallows: The proximity-focused image A consists of portions which are 
focused A(f) or unfocused A(b). The remotely-focused image B is similarly consisting of focused 
B(f) or unfocused B(b) parts: 

15 A=A(f)+A(b) (la) 
B = B(f) + B(b) (lb) 

The 'averaged 1 picture M is now created.: 

20 M - A + B - A(f) + A(b) + B(f) + B(b) (2) 

The defbcused average image M(b) is next created: 

M(b) - A(f)(b) + A(b)(b) + B(f)(b) + B(bXb) (3) 

25 

where (b) represents defbcusing/blurrmg and (f) stands for focusing ion accordance with what was 
written above. The following relationship applies more or less like a definition* to the transition from 
the state of optimal focus to a state of blur 

30 A(f)(b)«A<b) (4a) 
B(fXb)-B(b) (4b) 

The assumption that image information defocused twice is yielding the same result as if defocused 
once only, is an approximation. However we are nevertheless writing: 



35 



A(bXb) = A(b) (5a) 
B(b)(b)-B(b) (Sb) 
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(4) and (5) are now substituted into (3), giving 

M(b) = 2A(b) + 2B(b) (6) 

5 mi the intensity of this image (6) is finally halved and subtracted from the average picture (2), giving 
us the resultant picture R: 

R= A(£) + A(b)+B(f) + B(b) - M(b)/2 - A(f) + B(f) (7) 

1 0 This final image (7) may now be compared to the average picture (2) above: The unfocused image 
information A(b) and B(b), from original pictures, has apparently disappeared, while the focused 
image information is retained. Or in other words: Using this method, the image contrast has been 
enhanced by rejecting image-components which are out of focus, the in-focus information being 
retained however. As already mentioned, these relationships reflect an approximate model for 

1 5 defbcusing: Picture regions are rarely completely in focus or out of focus, rather something 

in-between. The discussion is nevertheless indicating a distinct possibility to cut down unfocused 
image components, from average images. These further processed images are henceforth called 
'contrast-improved average images T . The discussion (above) involves only two differently-focused 
images, however the discussion is valid for any number of pictures, not shown here due to triviality. 

20 These methods producing contrast-enhanced average pictures may be used for viewfinder 

applications, when making template pictures (described elsewhere in this text), for certain video 
camera- and such still photo camera applications, where the resultant image is deemed 'good enough' 
for its purpose, this latter property not always being the case however, 

25 3. filter-methods 

Each of the original pictures are, according to another method developed, filtered by means of 
laplacian or fourier operators (Cf. also the so-called Burt pyramid, American Patent 5,325,449 to 
Burt et at and American Patents 4,661,986 to Adelson and 6,201,899 to Bergen) whereby a aeries of 

30 transform-pictures are created. This filtering is executed row by row (filtering of video- and related 
signals), as far as these descriptions can be interpreted. Transform-images do generally consist of 
image-series (like L0, LI, L2, L3 ...Ll), containing, so to speak, all available image information as a 
whole, but where usually each individual transform image (like L0) is holding only part of the total 
information. The oharacter of these transform-images is entirely different from the original pictures 

35 and may therefore not be merged into a final image with improved depth of field. However their 
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intensity-distribution may nevertheless reveal such parts of a picture, which are more in tbous or 
(more appropriately expressed as far as lower order analysis goes)> where the largest concentration of 
outlines or edges in focus are located It's possible to map the high-intensity distribution of & 
transform-image (thus higher contents of focused image information), called sub-regions. Intensity 
5 distributions of such sub-regions on filtered images, up to a certain order (however usually restricted 
to lowest order(s), because of practical reasons ) are compared* enabling a selection of the 
corresponding regions - from the differentiy-fbeused, original pictures. Sub-regions of higher 
intensity, from the differently-focused and filtered images are thus identified by using this technique, 
and the identification serves (as far as filtered-Jmage intensity and optimal focus correspond to each 

1 0 other) the purpose of pointing out the associated sub-regions on original exposures, for a final image 
synthesis, with depth of field-improvements. This method may require respectable computing 
capacity, in case all transform images up to a certain order (i) are to be processed. There are 4 times 
more pictures to process with the transform pictures L0 9 LI, L2 and L3 than if only a single picture 
L0 is to be used. This is a reason why the laplace-filtering process is so often restricted to lower order 

1 5 analysis only, consequently (as far as the selection process goes) only utilizing a limited part of the 
total image information, from original photos. It's previously known that laplace-filtering, of this 
kind, Is suitable for identification and reproduction of edges and patterns, a frequently desirable 
property when working with microscopes, 

20 4. Segmental Methods (SM) 

Original pictures are electronically subdivided into sub-images or segments according to an aspect of 
the present invention, this being another further development, These pre-selected portions of the 
image are analysed as regards image resolution or other parameters. A choice of image parts or 

25 segments having superior image definition, from respective original images, may thereafter take 

place. These select segments are merged into a final image. The name 'Segmental Method* (SM) will 
apply here to this technique, It differs conspicuously from other techniques in that the segments are 
distributed all over the original pictures, before the main image processing starts. There is 
furthermore no need for filtering of original pictures and finally, as a result, the total image 

30 information is utilized when choosing the segments. These segments (i.e. sub-images) arc also the 
same or similar and evenly distributed over the picture areas, according to a preferred mode of 
operation. 

This method is therefore particularly suitable for the art of photography* where depth of field- 
improvements are aimed at, where a primary object of the photographer is to reproduce a scene as 
35 faithfully as possible. On the other hand, the purpose is not to enhance/extract certain details, like 
edges, contours or patterns. Similarities rather than structures or patterns are therefore searched for in 
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a preferred mode of operation, see below. It may furthermore be pointed out that segmental metbods 
are also distinctly applicable to other selection criteria than image resolution. 
It may prove advantageous, during ongoing image processing, to change shape, sise, position, oar 
combinations of these, for at least some of the segments subdivided, c£ also part #5 below for one of 
5 several possibilities, plus some relevant comments in part #1 0. 

To sum up, the original pictures are divided into sub-images (segments), whioh are compared and a 
subsequent selection from these image parts is then performed, according to applicable claims and 
descriptions of the present invention. These segments, selected from original images recorded, are 
merged into a resultant image with better depth of field-properties than each individual and original 
1 0 picture by itself. This can be done in many different ways, a representative selection of them 
appearing below: 

4a* Contrast methods 

1 5 This teohnique, belonging to prior art is utilized when adjusting for some advantageous focal 

distance, when taking single photos. The measurement may then be performed within a few picture 
areas, providing some farther optimisation. (For further references, see the periodical FOTO #1 1978 
and American Patents 4,078 a 17 1 or 4,078, 172). Segments with highest available image definition, 
may be identified, using this contrast measurement technique: The image contrast is generally 

20 increasing, as the image resolution improves. The contrasts of different sub-images are thus measured 
and compared, according to an aspect of the present invention, Those sub-images showing higher 
contrast and therefore - in general - have higher image resolution, are selected. All such segments, i.e. 
qualified according to this criterion, are thus selected to be part of the depth of field-improved final 
picture. This is consequently one of several selection methods, however already dealt with in this tract 

25 and since this measurement technique is already documented in other patents, it's not to be repeated 
here. Subheading for this segmental method, is the 'contrast method 1 , 

4b. The Template method 

30 The Template method 1 is a name coined for another comparative segmental technique, with the 
following characteristics: A differently produced, depth of field-improved photo (template), is first 
created far the purpose of comparison. This 'other' technique might be some averaging method, 
possibly contrast-enhanced, or any other segmental technique like the above-mentioned contrast 
method, and there are still many other ways to bring it about. The important thing is not how the 

35 template picture was produced, but rather that its adequate for a comparative procedure viz. towards 
the original photo recordings. The template picture is - again - subdivided into sub-images, same as 
for the original exposures. Corresponding sub-images from original exposures are now compared 
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with associated sub-images from the template picture and that original sub-image stowing greatest 
similarity with the 'same' template sub-image, is selected for the final assembly of a resultant and 
depth of field-improved picture. The 'similarity' can be estimated/calculated in many different 
ways. However, some kind of comparative score is generally set up, involving pixel values from 
original-photo sub-images, being compared to corresponding pixel values from the template; For 
example by using a suitable algorithm, subtracting corresponding pixel values of an original photo 
and the template from each other, thereafter calculating some power for these figures, finally adding 
or averaging these contributions to some kind of score for the whole segment. There is thus more 
similarity, when the accumulated difference (score) attains a low value. The present invention is of 
course not restricted to these particular examples. Distinctive features of the template method, as 
compared to other methods, may be summarized as below; 

1 . A field of depth-improved template picture is produced by other means, 
for the purpose of comparison. 

2. Original photo-segments are not compared to each other but are compared 
to segments from the template picture instead, 

3* Greatest similarity in-between picture parts from the original and template 

photos are identified by means of comparison. 

4. The Template method does not identify any segments with maximum 
contrast nor image definition as such. 

There is of course no requirement for the template picture to qualify as a final result - this would in 
fact make the comparative template method superfluous. However the template pictures must 
nevertheless have such qualities that the comparisons with original exposures indicate the correct 
sub-image choices. 

5. Segmental variation-methods 

Problems in common for the above-mentioned segmental methods include failure due to low contrast 
or picture areas lacking (sufficient) detail. Edges and picture-contours may furthermore cause 
disturbances (artifacts), particularly along focus-edges while similar disturbances may appear along 
sub-image borders. The selection-methods are statistically more reliable when many image points 
(pixels) are involved within a particular sub-image. On flie other hand, the selection process around 
focus-edges (i.e. edges which separate picture areas of differing states of focus) lacks precision when 
using large segments, due to the fact that such segments may land upon a focus-edge and as a result 
the segment-selection is, in such cases, bound to be in error for part of the segment, i.e. generally a 
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more severe error for the larger sub-images. The segmental variation-method, is her* tie name coined 
for a general technique, where the object is to reduce such defects. Its characteristics are a* follows: 
Pixel-contents of the segments are changed by means of modifying their sizie, shape and position, 
thereby generating new (statistical) basic data for the segmental methods just described. One 
5 preferred mode is to change size of rectangular segments (like 2x2; 4x4; $x8.,..nxn pixels), 

Vertical and horizontal translations of one or several pixel intervals or rows, of a whole predefined 
segment-web* is another mode of preferences, creating a sequence of differently positioned but 
otherwise similar segment-patterns. Some of the pixels, from each segment, will be replaced by other 
pixels from adjacent segments when performing these steps. However onSy a limited number of such 
1 0 web-translations are possible, without trivial repetition. For example; An ideal image without external 
boundaries is subdivided into segment squares (like lxl; 2x2; 3x3; 4x4 or ....nxn pixels), where the 
number of possible patterns N» without repetition of segment-contents, may be given as: 



15 



N-nxn (8) 



The number of unique 'different' sub-image web-positions is thus 4x4 =16, with the segment-squares 
sized 4x4 pixels, however all these permutations are not necessarily required for an image process. 
The selection procedure, according to any of the above-mentioned segmental techniques, may now be 
repeated as a whole for each of these web-positions and, as a result, several versions of a processed 

20 resultant image are created despite the feet that the same original exposures were the basis. For 
example, a pixel by pixel average from these resultant images may now be calculated, giving us the 
final image result; thus no longer founded upon a single 'decision 1 but rather upon a multitude of 
'decisions', based on the more balanced and complete statistics* created by die different segment- 
patterns. This averaging does not affect, alter nor modify image regions with a stable and 

25 unambiguous state of focus, corresponding to one original image only. And this is because the 
averaging process takes place after the selection procedure. On the other hand, only those image- 
regions are influenced, where the choice in-between different original images is unstable, because of 
various reasons such as vicinity to focus-edges; Or in other words: Wherever a change of segment- 
size and position may influence the segment ohoice> this however not being the most general case. 

30 Averages from these ambivalent special cases reflect the uncertainty. This segmental variation 
technique does furthermore cause boundaries in-between adjacent segmcnts/sub-images to change 
place. As a result, possible disturbances, discontinuities and other imperfections abate as the 
sub-images are moved around and averaged. A disadvantage of the segmental variation-technique 
might be if s time-consuming properties, due to repetition of the selection process. 

35 
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6. Edge-Methods 

Image edges or contours are of at least two different kinds: Those caused by contrasts, U. strong 
intensity gradients (named 'contrast-edges' here) ami those created by a boundary in-between image 
5 regions in different states of focus (named 'focus-edges' here). An edge may well be of both kinds, at 
the same time. As already mentioned, an ambivalent situation occurs whenever a segment falls upon a 
focus-edge, A way to avoid this is to find first those edges (for example with a kplacian analysis, 
already described) and then modify the sub-image division accordingly, wherever the sub-images 
happen to fall on such edges: For example by a further subdivision of the segments involved, into 

1 0 smaller sizes or by adjustment to more flexible shapes, so that these segments are distributed on either 
side of an edge, more or less. As a result, segment areas being influenced by focus-edges, are 
reduced. It? s sometimes possible to have sub-images follow the shape of an edge, 
A nearby focus-edges may, if being out of focus, obscure a background in focus, thus reducing image 
contrast along the focus-edge borders. This is essentially a perspective effect, as seen from the 

15 entrance aperture. The effect may be reduced by decreasing aperture, thereby reducing the width of 
this edge-zone. 

Another remedy is to introduce a certain (relative) amount of electronic or optical magnification for 
proximity-focused images, so that focus-edges of foreground objects expand and, as a result, cover 
those zones with reduced contrast, more or less. 

20 

7- Segmental Exposure Control (SEC) 

Another member of the image-improvement technique group, utilizing segments, here under the name 
'segmental exposure control shall now be described: A subdivision of original images into parts is 

25 presupposed even with this method. The purpose, according to another aspect of the present 

invention, is to improve the selection procedure for those picture areas, which would otherwise be 
over- or underexposed. The object is to control the exposures individually, Le. for different segments, 
thus avoiding under- or overexposures and ensuring registration of more detail within the different 
sub-images. As a result, selection-methods with reference to depth of field ore improved. Its a known 

30 fact that over- and underexposures of image-registering instruments (like film cameras, digital still 
photo and video cameras, image intensifier devices and infrared instruments), occur because the 
detectors (like CCD sensors, film emulsions or image cathodes), electronics (like AID converters) and 
presentation media (like image screens and photographic paper) can only detect, process, respectively 
present a limited range of intensity levels 1 (of bandwidth) of incident light, through the optical 

3 5 entrance aperture. Or in other words, the front optics of an instrument may well project almost 
equally good images, regardless of high or low light level, while detectors, electronic processors 
and presentation media are more restricted in this respect, suffering from reduced capacity to 
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represent the total light-intensity Bpan or interval. The initial detection is ot particular significance m 
that respect, because infbnnation being lost here, will irretrievably disappear even for the subsequent 
electronic process. 

However, an optimal exposure for each little part of the whole image may be achieved by means of 
individual control of each part of the scene, i.e. by a differentiated variation of exposed amounts of 
light As a result, each little part will be registered under a more favourable average intensity-level for 
the current sensor. The method is illustrated by Fig 4. 

Exposure control, according to this other aspect of the present invention does here, by definition, 
include a differentiated control of light-quantities exposed as well as spectral properties (white- 
balanceX the latter quality also being subject to differentiated adjustments during detection or image 
processing, so that locally conditioned and troublesome tint-aberrations within for example sky 
regions or shadow areas are reduced or eliminated. 

The varied and for some image-areas mixed lighting of light-sources with differing spectral character, 
like the sun or sky, incandescent or fluorescent lamps or flashlights may even create differing local 
states of white-balance on one and the same photo. The remedy is however a correction, using this 
'local 1 technique. This procedure, of 'segmental exposure control' is exemplified as follows: 

1 . . As a first step, the local average light intensity for each little picture part 

or segment, is measured by means of a sensor belonging to the instrument 
20 2. The scene is exposed hi such a way that each segment or individual 

picture-part is exposed/illuminated in the most optimal way. 

3 . Image processing, including possible depth of field-improvements 
according to the invention (c£ above), takes place. 

4. The differentiated image-intensities ate restored more or less, 

25 i.e. original (c£ step #1) average light intensities are recovered or, 

if applicable, are adjusted for presentation media. 

This last step #4 may involve a frade-of£ namely a compression of the restoration in such a way that 
intensity-variations involved may fit within some constrained interval or ^bandwidth 1 of the 
30 presentation- or memory media available, so that image detail associated with exposure-extremes are 
not lost This response may aim at a logarithmic or asymptotic behaviour, similar in character and 
function to an eye or emulsion-film. 

The method of segmental exposure control was created in order to improve on the segmental selection 
process, where saturation situations occur when registering segments. In other words, where segments 
35 would otherwise be over- or underexposed to such a degree that image detail and contrast projected 
by the entrance optics, is lost Cloud formations of a bright sky may for instance lade away 1 , or 
foliage inside a deep shadow may be 'absorbed* by darkness in the process of image registration. 

27 
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Finally a discussion in more detail about how this selective exposure (of. step #2 above) control may 
be arranged; Assuming (though the present invention not restricted to) a so called digital camera 
exposing more than one frame (like 2) of the some scene. First a standard exposure of - Bay - 
1/1 00 second, such as a "normal' (prior art) automatic exposure control would dictate. Secondly a 
5 (comparatively) much underexposed picture with - say - 1/400 second exposure time. We note that 
an extra 1/400 sec. exposure does not here significantly introduce extra motion blur as compared to 
the 'normal' 1/1 00 sec exposure. The exeoution may furthermore, in favourable cases, take place in 
fast succession, because no mobile components need to be involved The other parameters like 
focusing, aperture stop, focal length etc are here remaining the same, for the two exposures. The point 

10 is that (otherwise) overexposed picture areas (like bright sky of a landscape eceoary) are more 

appropriately exposed by means of the shorter exposure, The electronic camera processor may, after 
image registration, select such segments from either image, that are most optimal as regards exposure. 
And, because the sky is now retaining more detail on the frame subject to shorter exposure time* we 
may also expect the final picture to become more detailed. And as a consequence, it may be more 

15 reliably processed as far as depth of field-improving decision-methods of the present invention are 
concerned. 

This differential exposure-method using sub-images, may continue to function and yield enhanced 
image quality, related to the same exposure-control improvements, even when the bstruinent/caniera 
is restricted to register pictures, of one focal state only, Le. whenever the depth of field- improvement 
20 function, according to an aspect of the present invention, has been 'switched off. And thus at last, as a 
spin-off from this discussion: It's evidently possible to apply this SEC image improvement technique 
to other independent contexts, i.e. even where instruments/cameras are lacking these depth of field- 
improvement facilities altogether. 

The method does of course allow for more than 2 differently-exposed frames to be used, however 
25 practical limitations are there, as far as total exposure time is concerned and too many sequential 
and/or long exposure times may cause unacceptable motion blur at the end of the process. The 
method does also require more memory and calculation capacity, because more pictures must be 
processed as compared to 'classic 1 photography, according to present day technology and this does 
particularly apply to the combination with depth of field-enhancement imagtag~techniques already 
30 discussed. However the performance of electronic processors end memories are presently undergoing 
a fast development which will presumably favour the present invention. 

Related prior art and available technique, where the object is to achieve better exposure-control when 
taking a photo, include various kinds of optical filters (like for enhancement of cloud formations), 
camera settings for different types of scenes (snow-scapes, taking photos against the light etc) and the 
35 so called exposure-bracketing methods, where several differently-exposed photos are taken, hi order 
to facilitate identification and choice of a preferable exposure. They all have in common that one state 
of exposure only is 'allowed' for each single photo. The segmental exposure control method on the 
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other hand; as proposed here* involves several different states of exposure within one and the same 
final image-frame. 

Existence of image sensors, with the specific property to allow for local variation of exposure time, 
provides us with one more technique for applying the differentiated exposure teohnique: This is 
5 where exposure time may be chosen in situ on the sensor, already during the process of detection. 
A sensor with differentially variable sensitivity (the equivalence of light sensitivity for photo 
emulsion-films) provides a similar mode. Another technique is to exeoute several exposures, using 
same exposure time, followed by some appropriate, controllable addition of these 'contrifeutione 1 
(pixel by pixel and/or for each segment) until an optimal T add-on' average intensity has been reached 

1 0 for each individual segment. There is finally one more way to control the amount of light exposed, 
i.e. by means of changing the relative aperture of the projecting lens: Several different exposures with 
same exposure time but different P-number, may now take place. They are executed with different 
aperture settings though, for instance during the process of one single and continuous aperture 
reduction. Several differently-exposed picture-parts or segments are thereby registered and optimally 

1 5 exposed segments may be selected aB before, qccordrog to the principles already disclosed about the 
segmental exposure control technique. 

All these applications may well show variations of practical interest, they are nevertheless all 
embracing the same basic principles which characterise this method of differentiated exposure 
control, i.e, the different image areas or segments of which the whole picture is built up, are subject 
20 to individual exposure. 

8. Flash Photography 

The depth of field-improvement technique* according to the present invention, does also call for a 
25 more optimal exposure control when illuminating a scene by artificial means. It's a well-known fact 
that flashlight being used in photography, may severely flood the scene, 'eroding' the picture of 
foreground objects, still leaving the background utterly uniUumineted, with a pitoh dark appearance. 
This is due to the fact that light-intensity is quickly fading when receding from a light source. The 
exposure time is, according to well-known prior art, constituting an average of a sort, a compromise 
30 where certain objects of intermediate distance may be acceptably exposed while nearby objects 
become much overexposed and the background underexposed. The technique of exposure control, 
using segments, (cf. previous part #7) proves useful even for flash photography in combination 
with the depth of fieittimproving methods discussed. It's now possible, thanks to die differential 
exposure control, to choose in-between several differently exposed frames, for each state of focus ' 
3 5 (there being several or one only of the latter). For instance the following two exposures: First 

a 1/400 sec exposure, followed by a 1/1 00 sec standard one. At least part of the very foreground is 
T better r reproduced by the first (1/400 sec) 'shoe while the second 'normal' one should give optimal 
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exposure more or less, for intermediate distances. A state of focus corresponds to a particular focal 
distance, indicating m obvious possibility to link these two entities with a formula. And, assuming 
some pre-programmed knowledge about how the (flash) light-intensity is diminishing vs. range, we 
may let the camera calculate and decide about which one, of several differently-exposed frames, is 
5 most optimal for a certain focal distance. 

The following method with differently-focused exposures (associated with certain distances, 
of. above) and variable ittumination, is applicable for the most common pre-deteimined oases, where 
states of focus are known before exposure, mi where the illumination of a scene is essentially 
artificial by means of flashlight or some other controllable floodlight or similar device on top, or in 

10 the vicinity of the camera. It's furthermore known how light intensity is fading with increasing focal 
distance, thus enabling calculation of the most optimal average illumination-levels, associated with 
respective states of focus. The illumination device may for example be designed so that the amount of 
light can be varied by electronic signals or other means via the camm/instrumetnt, in such a way that 
the nearby- focused frames are exposed under less amounts of light, while the most distantly-focused 

1 5 images are exposed with mote or sometimes all available light, depending upon the actual focal 
distances. Optimal flash intensities and/or exposure times, are thus set by actual object distances, 
which in turn are occasioned by predetermined states of focus. Direct relationships in-between states 
of focus and optimal illumination-levels are thus established. The individual exposure control was 
here applied to each differently-focused image frame as a whole, while the object was to lose less 

20 image detail due to unsuitable exposure. As a final result, the depth of field-improvement techniques, 
where segment selection procedures apply, benefit from this technique, 

9. Depth of field reduction methods 



25 The opposite effect to a depth of field-improvement, Le. a depth of field-reduction, may prove useful, 
wherever the purpose is to enhance certain objects of the scene and where it is advantageous to 
suppress some annoying fore- or background, as may be the case in certain contexts. This process, 
aiming oppositely as compared to the before-mentioned depth of field- improvements, is nevertheless 
following same principles more or less, as evidenced by the following example: 
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1 « A 'priority-image' Q) is chosen by the operator. Objects being in focus, on this 

particular image, are to be enhanced. 
2. An initial segment-selection procedure, following part #4 (above) will now take place. 

Optimally focused sub-images will thiu be selected from the differently-focused 



3 . Those of the select segments (stop #2) and furthermore belonging to 0) &e priority- 

image only (step #1), are forwarded to a memory. 
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4a. A selection procedure f in reverse 1 is next performed with the remaining segments: 

The most unfocused/blurred segments are thus selected and forwarded to memory. On 
4b. A pixel by pixel summation- or some other kind of compound picture is made from the 

rest of the segments, optionally being subject to further blur by electronic means, 
5 finally forwarded to a memoiy, 

5. A resultant image (R) is assembled from the optimally focused segments, belonging to 

priority image Q), according to step #3, plus the blurred segment contributions 

from #4a/b, 

10 Steps #4a/b may be varied and combined in different ways. The feature in common however* for 
these procedures, is the principle of first selecting picture parts, optimally focused, from a certain 
pre-select priority-image, thereafter in the most expedient way, choose and/or blur the rest of the 
segments, hi order to degrade image definition for other regions of the composite final picture (R). 
This depth of field-reduction method may be regarded as a depth of field-filter, providing a variable 

1 5 depth of field restraint, around a priority-image: The priority state of focus (P) is surrounded on each 
side, by two diJSferently-focused states (P+ and P-X according to a preferable mode of application: 
Thus, the available depth of field- interval becomes narrower as the object distances related to P- and 
P+ approach the priority-distance of F s fitom either side. Even segments selected from pictures, 
associated with P+ and P- may have fairly good image definition as such, being taken from the 

20 neighbourhood of some priority object in focus more or less, nevertheless appearing 'blurred 1 on the 
final step #5 picture (R) , because of additional image blur being introduced by step #4a/b above, 
However, the two reference exposures P+ and P- should not be chosen too closely to priority-image P 
because the images would then become too similar and as a result, the 'decision process' according to 
steps #2-3 (above) would then suffer from a too high failure-frequency* This method is applicable for 

25 camerarviewfindcre, when performing manual focusing or when a photographer wants to concentrate 
his attention on certain objects* in other words become as little distracted as possible by image- 
sharpness variations of other objects within the field of view, It's possible> according to another 
application, to simply replace the blurred segments from step #4 (above), with a uniform 
monochromatic RGB signature, like blue, thus placing the select objects of priority against a 

30 homogenous background without detail. A de facto separation from the fore- and background of 
select objects in focus, from a certain image #j, has thereby taken place in this specific case* Its also 
possible to replace the blurred sub-images from step #4 (above) with the corresponding segments 
from an entirely different picture, answering to a known want within the moving picture art, to create 
special effects, separating or merging various scenes, 

35 
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10. Combination methods 

Conditions prevailing for Instruments and Cameras of the present invention, may vary considerably 
and particularly the scenes registered, exhibit such diverse character that it comes hardly as a surprise, 
5 if these methods proposed exhibit differing utility for various contexts Of applications. Even image 
processing, of one and the same picture, may improve if allowing tfaeae methods to work together, 
interacting in a spirit of using each method where it performs best* The contrast method for example 
is sensitive, thus suitable for sub-images of low contrast, while a template method may give fewer 
disturbances (artifacts), thus being more suitable for segments of high contrast The contrast- 

10 enhanced average method may prove more advantageous for a viewfinder, where image-quality 
conditions tend to be less severe, but where instead simplicity and speed are awarded. Plain 
summation- or average methods may be used whenever a viewfinder is purely optical and thus few 
other means are within sight, while apparently the segmental exposure control is most suitable in 
eases of large intensity variations across a scene (like when using flashlight or photographing 

1 5 against the light) and where (digital cameras) a considerable number of segments would be Saturated 1 , 
i.e. become over- or underexposed, if not using this technique. The segmental variation method can 
be used where the scene being reproduced is demanding, Le. 'problematic 1 in a sense that 
unacceptably high failure-frequencies result from single selection- or iteration-rounds. Finally, the 
depth of field-reduction mode, may prove useful for cameras when selecting priority-focus through a 

20 viewfinder, a procedure likely to precede some depth of field-improvement process. The way these 
different methods are united by means of writing macro programs (*.hat files etc) is such a 
well-known engineering technique thai there is no need here for repetition nor expanding upon the 
subject any further. 

A careful pixel by pixel alignment of the differently focused/exposed images is presumed for 
25 all these above-mentioned multiple exposure-methods, this being the basis for all pixel by pixel 

comparisons and superpositions in the process. Side translations, twist or tilt of the images projected* 
in relation to the detection surface of the sensors) must therefore not take place while focusing or 
performing other movements. This is a matter of mechanical stability and tolerances of the 
instruments and cameras involved, more or less. Residual alignment errors may still influence the 
30 electronic image registration, however these errors can be detected and corrected by other means like 
some image correlation program, readily available (being prior art, c£ Swedish Patent #8304620-1). 
All these methods and application examples presented, being subject to multiple exposures of the 
same scene, are fundamentally related, following same basic principles, i.e, the object of them all is to 
manipulate the depth of field for an image in a most efficient way. 
35 A final appropriate comment, concluding this survey of select-information processing, related to 

differently-focused/exposed original-image records, may therefore be that said methods, as described 
in abovo-mentioned parts #1-9, can be successfully combined in various ways. 
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CLAIMS 

What is claimed is 

1 . Aa electrooptical instrument having an objective lens for reproducing a scene composed of objects 
within a field of vfaw at different object distance* in front of said lens, afbcuaing device for setting 

5 said lens at various focal distances, at least one electronic image defector having an entrance plane for 
detection and record of image information corresponding to an image of said scene, an electronic 
image memory for storage of image information registered by said image detector and an 
image-shaxpne&B detector, characterized by: 

a/ The focusing device being arranged for simultaneous and/or time-sequential focusing of said 

10 instrument at different object distances* 

b/ the image detection being arranged in such away that imago information* corresponding to at least 
two difltottntiy-focnsed images, Le. with differing states of focus, is recorded and 
c/ means being assigned for having said image-sharpness detector geometrically and similarly 
subdivide said images into image parts or sub-images corresponding to each other in such a w$y that 

1 5 there are similar sub-images, from respective differently-focused images, depicting a similar part of 
said scene being reproduced and 

df means being arranged for having said image-sharpness detector directly or indirectly, and from 
each set of said corresponding differently-focused sub-images, select and forward to image memory* 
that sub-image information contributing to optimal image resolution, and merge said select image 
20 information from corresponding image parts into a final image having better image resolution than 
each individually focused image record by itself* 

2. An instrument of claim 1 characterized by means for measuring ranges with a range-finder, to 
parts of the scene in front of the objective lens. 

25 

3 . An instrument of claim 2 characterized by a focal distance-sensor being part of the range-finder, 
and where said sensor is interacting with said image-sharpness detector in order to register those 
ranges for each part of the field of view, giving optimal image definition, 

30 4, An instrument of claim 1 characterized by means being included with said image-sharpness 
detector, for detection of light intensity differences in-between adjacent zones of pixel detectors* 
located in each multiple of said sub-fields of view. 

5. An instrument of claim 3 characterized by means for range-*4justment registration, wfe related to 
35 said objective lens, and means for saving hi memory such object distances recorded, giving optimal 
image resolution for each image part subdivided, and this resulting in a three-dimensional record of a 
scene in front of said lens. 
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6. An instrument of any previous claim characterized by the said image-detector device having 
several parts, and they each record images being projected by said objective lens, i,e. depictions of the 
Batne scene, albeit differing as regards focusing. 

5 7. An instrument of claim 1 characterized by means being arranged for measurement of image 
contrast at each of said difierentiy-facused sub-images and means being assigned fat selecting 
and forwarding image information from such sub-images showing optimal contrast. 

8. An electrooptical instrument or oamera of claim 1 and said scene being illuminated with adjustable 
10 amounts of light by aa artificial source of light, characterized by means being assigned for control 
and variation of said iUuminatian in such a way that each image registration of said scene is subject to 
an illuradnation-intensi^ level, aiming at optimal exposure for that object distance linked to flie state 
of focus, associated with said registration. 

15 9, An electrooptical instrument of claim 1, characterized by means being arranged for production of 
a depth of field-improved template image, from said differently-focused images, this template being 
subdivided into image-parts or sub-images, like for said diflferendy-focuaed images, and means being 
arranged for having the image-sharpness detector measure a similarity in-between each of said 
corresponding differently-focused image parts on one hand and the template-image equivalent part 

20 on the other, then select and forward image information from such sub-images being most similar to 
the corresponding template sub-images, and merge this select image information, originating from 
each set of differendy-fbeused sub-images, into a final image with better resolution than each 
individually focused image record by itself. 

25 10. An instrument of claim 1 characterized by means being assigned for repeating said subdivision 
of differently-focused images once or several times, however each time with differing subdivision 
and/or differing size/shape of said image parts or sub-images, so that different resultant images are 
generated from one and the same scene depicted, and that a compound average-image of sorts is 
created, preferably by means of a pixel by pixel fusion/superposition of said resultant images, 

30 

1 1, An electrooptical instrument with objective lens for reproducing a scene composed of objects 
within the field of view at different object distances in front of said lens* a focusing device for setting 
said lens at various focal distances, at least one electronic image defector with entrance pkne for 
detection and record of image information corresponding to an image, i.e. the scene depicted, an 
35 electronic image memory for storage of image information registered by said image detector, an 
image-sharpneBs detector and an image-discriminatory device^ characterized by: 
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a/ the said Image-discriminatory device being arranged in order to enable a pre-selection of one or 
several priority-images, 

b/ the focusing device being arranged for simultaneous and/or time-sequential focusing of said 
instrument at different object distances, 
5 c/ the image defection being arranged in such a way that image infomration, corresponding to at least 
two differently-focused images, Le. with differing states of focus, is recorded, 
d/ means being assigned for having said image-sharpness detector geometrically and similarly 
subdivide said images into image parts or sub-images corresponding to each other in such a way that 
associated sub-images, from respective dtfiferendy-focused images* depict the similar part of said 

10 scene being reproduced and 

e/ means being arranged for having said image-sharpness detector, directly or indirectly from each 
set of said corresponding differently-focused sub-images, select and forward such image information, 
contributing to optimal image resolution in the first place and belonging to said priority-images 
selected in the second place and 

15 f! means being assigned for having the image-sharpness detector, directly or indirectly from said set 
of image parts, except for those parts already ohosen/forwarded from step e/, select and forward other 
sub-image information contributing to inferior image definition and 

gf means for optional image defbcusing being arranged, enabling possible further image definition 
degradation of said select image information from step if, and 
20 V means being arranged for assembling said image information from steps e/, il and optionally g/ to a 
final imago having inferior depth of field-properties than each individually-focused image record by 
itself. 

12. An instrument of claim 1 1 characterized by said image parts from step ff being exchanged for 
2 5 corresponding image-parts from another image, 

13. An electrooptical instrument or camera with objective lens for reproducing a scene composed of 
field of view-objects in front of said lens, an adjustable exposure device for setting the state of 
exposure, as defined by exposure time, relative aperture and sensor sensitivity in various 

30 combinations, and at least one electronic image detector with entrance plane for image detection, 
v/x image-information corresponding to said scene being recorded, and an exposure meter, 
characterized by: 

a/ means for subdivision of images recorded and of associated detector plane, info patterns of 
sub-images vis. corresponding sensor-parts and 
35 b/ means for having said exposure meter measure and register, for each said sub-sensor area, the light 
intensity projected by said objective lens, thus enabling estimates about which states of exposure are 
generating the most optimal light projection an each individual sensor-part and 

35 
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a means for exposing each of said sub-images, thereby registering each port individually, under a 
state of optimal exposure more or less according to step hi decisions for each image part and 
d/ means for performing electronic image processing including restoration of originally projected 
light levels on respective sensor-parts and, if applicable, limited capacity of memory- and/or 
5 presentation media necessitating further actjustments. 

14. An electrooptical instrument or camera of claim 13 characterized by an image detection with 
bJ means being arranged for recording image information corresponding to aft least two 
differently-exposed images, i.e. with differing states of exposure, 
10 b/ means being arranged for performing said image subdivision similarly for the different images, 
in such a way that corresponding sub-images from each differently-exposed image, depict the similar 
part of said scene reproduced and 

d means being arranged for said exposure meter to select and forward from each set of said 
corresponding albeit differently exposed sub-Images, that sub-image information corresponding to the 
1 5 most optimal states of exposure, i.e. contributing to the best detail-reproduction in the process of 
image registration, and merge this select sub-image information from each set of corresponding 
sub-images into a final image having better image detail-reproduction than each individually exposed 
image record by itself 

20 1 5. An electrooptioal instrument or camera of claim 13 characterized by further means being 
arranged for performing an image processing of whichever claim M2 and 17 s creating 
depth of field-improved final images. 

16. An electrooptioal instrument or camera of claim 1 3 ( characterized by means being arranged for 
25 an image processing where colour- and/or white-balance of said image parts or sub-images of the 

resultant image, are adjusted individually. 

17. An electrooptical instrument or camera having an objective lens for reproducing a scene 
composed of field of view-objects at different object distances in front of said lens, a focusing device 

30 for setting said lens at various focal distances, at least one electronic image detector having entrance 
plane for detection and record as image information of the scene, v& as a picture, and an electronic 
image memory for storage of image-detector information being registered, 
characterized by 

35 a/ means being assigned for simultaneous and/or time-sequential focusing of said instrument at 
different object distances and 
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bf image registration being arranged in such a way that image-information, corresponmng to at leasi 
two differently-focused images, ie, with differing states of foous, is recorded and 
c/ Hie differently-focused images being arranged for superposition into a compound image (M) and 
d/ means being arranged for defocusing said compound image (Mb), and 
5 ef image intensity of said (Mb) image being arranged for a pixel-by-pixel reduction, by a factor k 
(Mbk) and finally 

f/ apixel-by-pixel subtraction of said image (Mbk) fiom said compound image (M) being airanged, 
giving a resultant final image (S) having better image definition and depth of field-properties than 
each individually focused image record by itself. 

10 

18. A method for photographing a scene in front of an objective lens of a camera, recording taking 
place via a detector-plane and registration being executed with the camera set to a plurality of 
differing focal distances, characterized in that the differently registered records are zonally 
subdivided in a mutually similar way, bo that for each zone is arising a plurality of sub-imagea 

1 5 corresponding to said plurality of focal adjustments, and in that a final image i s assembled by means 
of selecting that sub-image from the sub-images in each zone, showing the best image definition, 

19. The method of claim 18 characterized In that said recordings are executed in one context while 
said subdivision into sub-images and assembly of the same sub-images is taking place in another 

20 context and at another place. 

20. The method of claim 1 8 characterized by a registration-procedure where a firstly-registered 
image record is transferred to an image-memory, and a subsequent recording is compared sub-image 
by sub-image to the one present in the image memory B whereby that sub-image of the two having the 

25 better image definition, is selected, being retained in the image memory, and said method-steps 

being iterated for subsequent images recorded, so that the resulting final image ends up in said image 
memory. 

2 L An electrooptioal instrument or camera having a depth of field-modifying function, with objective 
30 lens for depicting a scene composed of field of view-objects at various object distances 

in front of said objective lens, and having a focusing device for setting said objective lens and 
having an image-registration device, characterized by: 

a/ said focusing device being arranged for focusing the instrument at differing object distances, 
b/ said image detection being airanged in such a way that image information, equivalent to at least 
35 two differently-focused images, ie, having differing states of focus, is recorded and 

of means being assigned for using differing states of magnification when performing image 
registration and/or processing of said differently-focused images. 
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